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Queueing Analysis of a Canonical Model of Reed-Time 

Multiprocessors 


ABSTRACT 

Multiprocesso"*! are beginning to be regarded increasingly 
favorably as candidates for controllers in critical real-time control 
applications such as aircraft. Their considerable tolerance of com- 
ponent failures together with their great potential for high 
throughput are contributory factors. 

In this paper, we present first a logical classification of mul- 
tiprocessor structures from the point of view of control applica- 
tions. We point out tha*. one important subclass has hitherto been 
neglected by the analysts. This is the class of syst. 2 ms with a com- 
mon memory, minimal interprocessor communication and perfect 
processor symmetry. 

The performance characteristic of the greatest importance in 
real-time applications is the response time distribution. Indeed, we 
have shown in a separate paper [2] how it is possible to character- 
ize rigorously and objectively the performance of a real-time mul- 
tiprocessor given the application and the multiprocessor response 
time distribution and component failure characteristics. We there- 
fore present here a computation of the response time distribution 
for a canonical model of real-time multiprocessor. 

To do so, we approximate the multiprocessor by a blocking 
model and present a means for efficient analysis. Two separate 
models are derived; one created from the system’s point of view, 
and the other from the point of view of an incoming task. The 
former model is analyzed along largely conventional lines. For the 
latter model, an artificvil server is used, and the system 
transformed into a queueing network. Validations show that the 
approximation is good. 
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handled by an Immediate extension of this approach. To solve this problem, we 
employ artificial servers. Artificial or fictitious servers have been used in a 
number of models. In [6], they are employed in the analysis of an open queueing 
network with blocking.' Our analysis, although superficially similar, is non- 
iterative and treats what amounts to an M/M/c queue with blocking. In such a 
setting, the Iterative technique presented in [6] is not applicable and the artifi- 
cial server approach must be combined with some means of determining the 
state of the system. 

We begin by noting that the system as shown in Figure 2 can be approxi- 
mated by that in Figure 3. The approximation is justified since the time spent by 
a task receiving service (as distinct from waiting in the queue) at the dispatcher 
is negligible compared to the memory service time. According to our approxi- 
mate model, then, an incoming task waits in the Memory-Dispatcher (M.D.) 
queue, and, provided it is admitted (see below for definition), receives service 
first from the memory and then is assigned a processor for execution. Tasks are 
processed by the M.D. only when the number of tasks already admitted is less 
than the number of processors. A task is said to be admitted when it is either in 
the process of accessing the common memory or executing at a processor. The 
Memory-Dispatcher waits for a processor to become free, then transfers the 
relevant software to the target processor. 

To analyze this model, we portray it from two complementary points of 
view. The first, which is the system-oriented model is as in Figure 3. On the other 
hand. Figure 4 depicts the system as seen by an incoming task, i.e., it is a task- 
oriented model. We consider each of the models in turn. The system oriented 
model will first be analyzed to provide parameter values that will then be used in 
the task-oriented model to obtain task response time distributions. 
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ences the architecture of the Interconnection structure. 

How well a multiprocessor controller performs clearly depends on the 
requirements of the application. We showed in [IbJ how controller performance 
could be rigorously and objectively evaluated, given the control application and 
multiprocessor response time distribution. Keeping this foundation in mind, we 
evaluate in this paper the performance of a canonical model of multiprocessors 
for real-time control. 

This paper is organized as follows. In the following section, we classify real- 
time multiprocessors into three basic logical families. Then, in Section 3. we 
analyze an important subclass of multiprocessors, obtaining an approximate 
expression for its response time. We conclude in Section 4 by describing briefly 
some extensions of this work presently being undertaken, and their implications 
for real-time multiprocessor design. 

2. Canonical Models of Real-Time Multiprocessors 

The execution by a processor of any task involves, first, the transfer of all 
or part of the relevant software to the processor private memory or cache, and 
then, its execution. Because the entire set of programs that the control system 
will ever execute is well-defined, the designer has an option: the system can 
either have a common, or mass, memory that contains all the applications 
software — thus requiring the transfer of the relevant software upon a task 
trigger* -- or every processor can hold in its private memory all the applications 
software it will ever need. The latter alternative eliminates the need to transfer 
software, thus reducing overall response time. Choosing this alternative 
presents the designer with two sub-alternatives; either provide each processor 


1. The operating environment is modelled as the source of a stream of task triggsrs, i.e., en- 
vironmental stimuli that cause a p^urticular set of tasks to be performed by the multiproces- 
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with a private memory so large that the set of control programs required in the 
control of the process can be held in it in its entirety, or preallocate tasks to 
specific processors. 

Based on the above observation, the following can be considered to be 
canonical logical models of multiprocessors used in real-time control. 

Type 1: A Type 1 multiprocessor controller is one in which the processors do not 
specialize; in other words, tasks are not reserved for specific processors, and 
each processor in the system may be allocated any task. The process of task 
allocation is typically dynamic. This type is divided inuo two subclasses depend- 
ing on the size of processors’ private memory. In a Type la system, the private 
memory of the processors is large enough to hold both the applications and exe- 
cutive software in its entirety®. This system is ideal for applications in which a 
small number of relatively independent and small tasks form the job mix. One 
example of such Type la is the CIl^FCS [8] system of the United States Air 
Force. 

In a Type lb system on the other hand, the private memory is too small to 
hold all the applications software and requires the transfer of software with each 
task trigger. This transfer can be either single, or involve paging. 

One can see the tradeoff between requirements of memory size and task 
execution time from this subclassification. As a general rule, the processors 
allocated a task execute it in its entirety without interruption unless some of 
the processors fail. There is thus little or no Interaction between the individual 
processors in this type of system. 

The best known implementation of the Type lb structure is the Draper 
Laboratory’s Fault-Tolerant Multiprocessor (FTMP) [5]. 


2. With memory densities rising and costs falling, this may be no longer as difficult as it once 
seemed. 
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7Vp« 2: In a Type Z system, the processors are preallocated specific tasks (or 
subtasks) and the software related to these is loaded in their private memory. 
With the identification of specific tasks with particular processors comes the 
problem of reallocation of tasks on processor failure. Thus, the system of this 
type is inflexible to reconfigure itself upon failure but its reliability is obtained 
in general through physical redundancy in system components. In general, Type 
2 is used when the individual tasks have significantly different time /safety criti- 
cality (e.g. flight control and navigation tasks in aircraft applications). Proces- 
sor interaction can be considerable. The SIFT [9] system built by SRI Interna- 
tional is an example. 

Type 3: A Type 3 system is a composite C a Type 1 and a Type 2 system. 

Figure 1 shows graphically the above classification of multiprocessors. 
Notice that the classification is logical and is different from most conventional 
ones that are usually based on physical interconnection structures. 

A considerable body of literature has developed around the problem of 
analyzing multiprocessors. Almost invariably, the procedure is to use a Markov 
model of the system to solve for such parameters as throughput, reliability, 
availability, etc. Type la systems have been the focus of a great deal of atten- 
tion. The tendency of almost all authors is to assume identical processors and an 
identical exponential service rate distribution for all job classes, upon which the 
system degenerates into an M/M/m queue. This analysis is then embedded in a 
determination of the multiprocessor performance. One good example is the 
work on clpsed-form estimations of performability by Meyer [3]. 

Type 2 systems can be modelled as queueing networks, and there is a large 
body of literature on this topic. 

Type lb, however, has been almost totally neglected. This is odd. 
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considering that one of the few multiprocessors actually to be conshructed (e.g. 
FTMP[5]) is an example of this type of system. Also, it is likely that Type lb sys- 
tems will grow in importance as time progresses since it is ideal when the job 
mix is composed of a large number of tasks, each called relatively in/t^quantly. 
Again, the analysis of Type 3 systems requires, as a prerequisite, the analysis of 
Type 1 systems. In this paper, we shall analyze an important subclass of the 
Type lb system. 

3. Response-Time Analysis of Type lb l^stem Without Paging 


3.1. Description of the Real-Time Multiprocessor 

The multiprocessor we analyze is shown in Figure 2. It has a dispatcher allo- 
cating tasks to c identical processors®. Service at the dispatcher is FCFS, and 
all the tasks place an identical (statistically speaking) demand upon the compu- 
tational resources of the system. 

When a processor is assigned a task, it first sends to the common inass 
memory for the relevant applications software. This is the only reference the 
processor makes to the common memory during a single execution. Tasks 
arrive at a Poisson rate of X. and all service rates are exponential (in Section 3.5 
we also consider non-exponential service rates): at the processors it is /i, and at 
the memory the software transfer rate (per task) is fj-m- Note that fj, represents 
the actual execution rate for an individual task on a single processor; it does not 
take into account software transfer time. 

A point tacitly made in almost all work in this area is that the input process 
is Poisson, and that all service rates are exponential. The principal reasons for 
such an assumption are that (i) while both input and service distributions are 
general in practice, they can usually be approximated well by Poisson and 


3, Since processors may fail during a mission lifetime, c is a random variable. 


- 6 - 


0RK3INAL PAGE IS 
OF POOR QUALITY 

exponential assumptions, and (11) that analysis of a system with general arrival 
and service distributions Is almost Impossibly difficult. 

The first point ensures that the exponential assumption leads to at least an 
approximate model of reality. It is important, however, to check this fact for 
each particular model by employing alternative approaches, eg., simulation. In 
Uiis paper, we carry out this task by means of a simulation program that 
assumes Weibull service distributions. The exponential distribution is a special 
case of the Weibull. By varying the standard deviation of the service distribution 
while keeping the mean constant, we obtain an indication of the range of input 
intensities for which the exponential assumption is a good approximation. 

It is readily seen that in this model, there is the problem of simultaneous 
possession of resources by the tasks. That is, there is a period when, immedi- 
ately after being allocated a processor by the dispatcher, the processor queues 
up for service at the common memory, for the applications software. During this 
period, the processor is effectively locked out. i.e., it is forced to remain idle. 
Note that the present system is assumed to admit of no multiprogramming. 

Since there is a period when a task is in possession of both a processor and 
the common memory, this multiprocessor does not fit into any of the well- 
behaved queueing models (such as M/M/m, G/k m, etc.). There are two 
approaches to obtaining a solution for the response time distribution. The first 
is iterative and employs the method of surrogate servers as in [4]. The second, 
which we present here, also involves surrogate or artificial servers but is non- 
Uerative in nature. It consists of approximating the multiprocessor by a model 
in which simultaneous possession of resources does not occur, but where block- 
ing takes place. We have thus translated our problem into the context of a 
blocking model. An important consequence of this, which we shall discuss in 
Section 4, is that the analysis of the case where paging is allowed for can be 
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handled by an Immediate extension of this approach. To solve this problem, we 
employ artificial servers. Artificial or fictitious servers have been used in a 
number of models. In [6], they aire employed in the analysis of an open queueing 
network with blocking. Our analysis, although superficially similar, is non- 
iterative and treats what amounts to an M/M/c queue with blocking. In such a 
setting, the iterative technique presented in [6] is not applicable and the artifi- 
cial server approach must be combined with some means of determining the 
state of the system. 

We begin by noting that the system as shown in Figure Z can be approxi- 
mated by that in Figure 3. The approximation is justified since the time spent by 
a task receiving service (as distinct from waiting in the queue) at the dispatcher 
is negligible compared to the memory service time. According to our approxi- 
mate model, then, an incoming task waits in the Memory-Dispatcher (M.D.) 
queue, and. provided it is admitted (see below for definition), receives service 
first from the memory and then is assigned a processor for execution. Tasks are 
processed by the M.D. only when the number of tasks already admitted is less 
than the number of processors. A task is said to be admitted when it is either in 
the process of accessing the common memory or executing at a processor. The 
Memory-Dispatcher waits for a processor to become free, then transfers the 
relevant software to the target processor. 

To analyze this model, we portray it from two complementary points of 
view. The first, which is the system-oriented model is as in Figure 3. On the other 
hand. Figure 4 depicts the system as seen by an incoming task, i.e., it is a task- 
oriented model. We consider each of the models in turn. The system oriented 
model will first be analyzed to provide parameter values that will then be used in 
the task-oriented model to obtain task response time distributions. 
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Th. quMUUet th.t thli modal prondo. u< with an |p,jj whan wa daHoa p,j 
to ba tha prababUity that than an I taakt In tha M.D. quaua (Ineludlni tha ana 


ncal«n« aanle. at m.mory It no blaakln* 1. takln* placa), and that J pracaa- 
ion aro axaeutln* (l.e , the appUcaUani saftwan hai baan tranifamd to than J 
praaaaian and actual axaautlan li takln* place). Keep In mind that blackln« at 

incamln* taaka ba|lni tc occur when l*J=c and a new task arrives «tare any 
admitted task has been completed. 


Nate that unless otherwise stated, a variable is hencetorth assumed to be 
sera if one or man at its subscript indices became nejative. The state- 
transition diagram appears as Fijun 5 and assumin, steady-state, th. tollawin* 
global balance equations can be written down; 


^Po.o = MPo.i 


( 1 ) 

(^+jM)poj = 0 + 1)mPo.jm + MmPi.j -1 

for j<c 

( 2 ) 

(X+c/i)po,c = AtmPi.e -1 


( 3 ) 

(^■•■/*in)Pi,o ~ Xpj_|,o+MPu 

for i>0 

( 4 ) 


for 0<j<c, i>0 

( 5 ) 

(X+C/i)pj e = Xpj_, e+/imP,+, e_, 

for i>0 

( 6 ) 

The boundary condition is. 



2 tp,., ‘ 1 

l»0 J«0 


( 7 ) 

To solve for p,.j. we use the method of generating functions. 



l^fine 
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Using generating functions and manipulating, we obtain the following rcoursion 
for o>l: 


gi(z) 


- :«o(z) - ~Po.p 


(9) 


gj4-i(z) - 8j(z) “ zmO+I) 


M(J+1) 

Mm 


Mm 


2M(J+1) 


for 0<j<c 


( 10 ) 


^ z[X(1-2)+cm] ~ z[X(1-z)+cm) 

From (9) and (10), we have 

gj(z) = Aj(z)go(z) + §Bj,,(z)p„., forCXjsic (12) 

IsO 

where the coefficients are defined by the following recursion for 1 s j < c and 
i<j-l: 


Ao(z) = 1 

» _ A(l-z)+Mm 

A.(j) 

A - A(l-z)4 -jM»Mm . Mm . . X 

M(j + 1) ZM(j+l)^^"^^^ 

■ M(j^-i) ' i^IoTT) 

Ft /_> - ^( ^ ~^) ■*■ JM+Mm „ y_\ . Mm 


(13) 


In particular, 


c-l 


ge(z) = Ac(z)go(z) + 2 Be,jPo.i 
i>0 

Equating the right-hand sides of (11) and (14), we have: 


(14) 


«o(z) = 


D(z)Ac-i(z)-Ac(z) 


-'I 

f^1 


i*0 


Be.i(z)-D(z)Be-i.((z) 


Pe,i 


Be.e-l(z) + D(z) 


Po.c-1 


(15) 


10 - 


where: 
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The only remaining imknowna are the poj for The equations 

required to solve for these are derived as follows. 

The boundary condition (7) yields the relation: 


£gj(l) = 1 (17) 

l«o 

Also, the generating functions gj(z) must converge in the unit disc |z|«l. All 
poles of the generating functions lying in the unit disc must therefore be can- 
ceilbd out by corresponding zeros. The use of this condition yields further equa- 
tions in the pe.i for 0«isc-l by (15) since the ieros of the generating function 
are functions of the values taken by the po.|. Note that no additional equations 
can be obtained from a further invocation of (10) since the apparent additional 
pole at the origin in (10) is cancelled out by a zero. 


It can be shown that the above equations are sufficient to permit the com- 
putation of the po,i values. The generating functions gj(z) are therefore com- 
pletely determined. By inverting them (numerically, in most instances), the 
steady state probabilities { pg | can be obtained. 

Recall that the above analysis is for c>l. For the case when c=l, we have: 


go(z) = 


zfimajz) - 
za{z)aM - MMm 


and 


«i(z) = 


*m(^) _ Mm _ 

""T — go(z; — jrPo.o 


where 


a(z) = \(l-z) + 
«m(z) = Ml-z) + Mm 


and 
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R&mark: The dispatcher queue in physical Implementations is clearly finite. By 
making it sufficiently large, say N, however, it may be considered etfectiveliy 

infinite with probability ^ f)pij. 

laO JaO 

3.3. Taek-Oriented Model 

In this model, we employ an artificial sarver to account for blocking delay. 
This yields an approximate solution for the response time distribution. This 
solution gives an upper bound for the true response time distribution, so that 
we err on the side of safety. 

An incoming task views the system es expressed in Figure 4. With a proba- 
bility a, it is blocked. The blocking is expressed in terms of an artificial server, 
to which the incoming task branches if it finds the system blocked, l.e.. the ser- 
vice rate distribution of the artificial server is the same as the departure rate 
distribution for the tasks executing on the processors. When the servipe rate at 
the processors is exponentially distributed, the artificial server also has an 
exponential service rate distribution. 

Implicit in this analysis is the assumption that steady-state exists at all 
times. This is patently not true. In most cases ~ considering that controllers of 
this type are generally lightly loaded — the probability of blocking is very small. 
However, once a blocking cycle begins, the probability of its continuing is 
greater than a for obvious reasons. Even so, it is unlikely that steady-state in 
either the hlocki ig or nonblocking mode will be established perfectly. This is 
especially the case when the input intensity is neither very low nor near to driv- 
ing the system into saturation. When in this intermediate range, one would 
expect the system to switch from the blocked to the unblocked state and vice 


P04) “ 1 “ “ + •“ 
^ M Mm 
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/ersa with a fairly high frequency. Hence, any model that is based on the 
steady-steady state assumption would be expected to provide less accurate 
results in the intermediate range of input intensity than in the extreme ranges. 
(Indeed, the relative accuracy between analytical results at two different inten- 
sities might be used to obtain some indication of the frequency with which the 
system switches from the blocked to the unblocked state). 

If we assume that steady-state exists at all times, the problem lends itself 
to a particularly simple solution and we obtain an upper bound to the true solu- 
tion. It will be shown by simulation that this upper bound is good. 

Under the assumption of steady-state, the system may be regarded as the 
queueing network shown in Figure 4. The artificiai server represents the block- 
ing delay. Notice that unlike in .nost analyses, incoming tasks have to wait in 
line to be served by this servti-. The interpretation of this last statement is as 
follows The task at ^he head-of-the-line (H.O.L.) position is waiting for the first of 
the tasks admitted to the system to leave sc that it can be unblocked and begin 
to attempt to access the M.D. Tasks that have entered the queue, after the 
H.O.L. task have this latter task as an additional impediment to entry into the 
system proper. The system represented in Figure 4 follows immediately from 
the above argument. Notice that under the queueing rules for this model, there 
is no queueing for the processors once memory access is completed: a task is 
admitted into the memory only when there is a free processor ready and waiting 
for it. 

It only remains for us. then, to compute the value of a and the artificial ser- 
vice rate, 

It is easy to see that 

« = 1 - S Pi.i UB) 


and 


A*m = 


PAGE fS 
OF POOR QUALITY 


(19) 


E iPPy 


S Pij . 

If Rc(s) is the Laplace transform of the response time distribution for the 
tasks when the system has c processors functioning, we have: 


R^(s) = L-^^+l-aL-^^523 ^ 

[ s+A^ j s+fi^ s+At 


( 20 ) 


where 


/A® “Mm (^l) 

and 

Mmq “ Mm ^ 

which on inversion yields the response time density function for c>0: 

rc(t) = Ae"'*»‘ + Be''*™»^ + Ce-*^. (22) 

where: 


^ — Q^MMaMriq 
” (Mm,-Ma)(M-M«) 
g _ M/*mq[M«~Mmq( 1 ~<^)] 
(M«~Mmq) (M“Minq) 

^ _ MMmq[Ma-M(l-«)] 
(M.-M)(Mmq-M) 


(23) 


3.4. Validation 

In an attempt to validate the analytical results obtained above, a multipro- 
cessor with three processors was considered. 

As a reference, a GPSS simulation model was developed for the system. The 
input intensity was varied over a wide range and results obtained through simu- 
lation compared with those obtained from analysis. In Table 1. the mean 
response times from the simulation and the analysis are compared. 

Throughout the range of intensities studied, the analytical and simulation 
results were acceptably close (lo within 10% in most cases). Notice that, as 
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predicted in the preceding section, the analytical and simulation results for the 
lower and higher ranges of the input intensities agree much better than those 
for the intermediate region. In particular, the agreement at the upper and lower 
ends is better than 9%. while that in the intermediate range deteriorates to 15%. 
The most probable reason for this is that in this range of intensity, the system 
switches frequently from the blocked to the non-blocked state and vice versa. 
Steady state is therefore not achieved in the intermediate caises and one of the 
assumptions inherent to the task-oriented model is thereby violated. In support 
of this, note that the analytical results for this region indicate a consistently 
larger mean response time than the simulation results. 

3.5. Non-exponential Service Rates 

While the extreme complexity of analyzing systems with blocking and gen- 
eral service distributions forces the analyst to assume exponential service dis- 
tributions. it is common knowledge that programs on many occasions have non- 
exponential service requirements. This can be modelled by equating the meaui 
of the actual service time to a fictitious exponential parameter. In such 
instances, the sensitivity of the model to this inaccuracy is crucial*. 

We test the robustness of our model to non-exponential service rate distri- 
butions by assuming those to be Weibull, which is a more general distribution 
than the exponential. This distribution has the distribution function 
Fw*(t)=l— The mean is 

^ = r((q+l)/q)/C 

and the variance is 

aS. = [r((rj+2)/q)-r®((q+l)/77)]/^ 

4. Indeed* the success of such models as the central server is due in a large measure to a re- 
lative insensitivity to this inaccuracy. 
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The exponential and the Rayleigh distributions are special cases of the 


WeibuU. obtained by setting and 17=8. respectively. When t 7<1, the variance 


is greater than the mean, when 17=1, the variance equals the mean, and when 
i7>l, the variemce is less than the mean. 


Our test for model robustness takes the form of plotting the ratio of the 
mean response time when rj is 0.5 and 0.7 respectively to when rj -1 (i.e.. the 
exponential distribution). The mean response time stays close to the exponen- 
tial value for a relatively large value of nominal input intensity. As expected, at 
high intensities, there is a marked divergence from the value predicted by the 
exponentied distribution. However, in critical control applications, the utilization 
is almost always very low to allow sufficiently broad margins of safety. Note that 
the actual input intensity is greater than the nominal value which defined in Fig- 
ure 6. We have performed computer simulation and obtained Figure 6. 


4. Di«:ussicm 

In this paper, we have presented a logical taxonomy for multiprocessor sys- 
tems used in control applications, and analyzed one important class. We con- 
sider here methods of extending and using this analysis. 

Extension to Type lb systems with paging, although nontrivial, is not diffi- 
cult. One would have here three classes of jobs: one in front of, or being served 
by, the memory-dispatcher, one undergoing service, and the third class consist- 
ing of jobs either being served by the memory after receiving a portion of their 
service at the processor or waiting to reaccess the processor. It is not difficult 
to think of a number of different systems based on variations of the above basic 
theme. For instance, one might obtain new models based on assigning priorities 
to the tasks. In all these cases, the artificial server approach can be used to 



advantage. 
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Also, the reader should bear in mind that we have analyzed a logical family 
of computer systems. This analysis is therefore valid for a wide range ot physical 
implementations. 

The response time distribution, once obtained, can be used in a number of 
ways. As already mentioned, we showed in [2] how response time distributions 
together with a probabilistic model of the computer and a full mathematical 
description of the control application can be processed to provide rigorous and 
objective means for evaluating the performance of a multiprocessor system in 
the context of its application. 

Tradeoffs can also be studied and the multiprocessor system refined 
thereby. For example, one might consider trading off processor number against 
processor power^ 7] . 
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